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ABSTRACT 

This guide provides a brief overview of the issues 
involved in the validation process. It was developed for evaluators 
of adult education projects, such as 310 projects. The guide is 
organized in three sections. The first section discusses the 
rationale for validation. The second section describes the 
characteristics of validation models. The validation processes 
described are the Joint Dissemintion Review Panel Process and the 
Identification, Validation, Dissemination process. The final section 
is an overview of other issues in validation, including adoption 
versus adaption, evaluation experience of project staff, exemplary 
versus promising programs, educational versus statistical 
significance, responsibilities of validated program staff, and costs 
of validation. The guide also includes references and submission 
guidelines for the Joint Dissemination Review Panel Process and the 
Identification, Validation, Dissemination Process. (KC) 
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INTRODUCTION 



The various state education agencies each year sponsor special 
demonstration and teacher training projects in adult education in accord- 
ance with provisions of the Adult Education Act, P.L. 91-230. Under this 
act, the purpose of Section 310 projects is to encourage the development 
of new approaches and models designed to address critical educational 
needs which have been identified as state or national priorities for adult 
basic and/or secondary education. These projects often involve the devel- 
opment, testing, and implementation of innovative ideas and procedures to 
help meet major needs in adult education in the state. They focus on 
potential solutions to common problems rather than specifically addressing 
unique local needs. In particular, the New Jersey Department of Educa- 
tion (NJDE), through its Division of Adult Education, aims for adult 
education 310 projects to result in replicable models that incorporate 
demonstrably effective products or practices which can be adopted or 
adapted by other educational agencies across the state. 

In a time of diminishing federal and state resources for local edu- 
cational initiatives, the identification, validation, and dissemination of 
exemplary programs becomes very important. Through sharing successful 
programs, practices, and products that can be disseminated, costs of 
needlessly reinventing solutions can be eliminated through systematic 
adoption or adaption of existing validated programs. For a program to be 
validated, it must demonstrate in some objective manner that whatever 
claims it has made for its effectiveness are indeed valid or true. In 
short, to be validated a program must prove that it does what it says it 



does. 
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This Guide provides an overview of the steps and concerns that 
projects must address in going through the validation process and briefly 
sununarizes some of the issues involved in the validation of exemplary 310 
projects. Separate sections of this Guide discuss the rationale for 
validation; describe the characteristics of existing validation models 
such as the Identification, Validation and Dissemination (IVD) process 
used by many states and the process used by the Joint Dissemination Review 
Panel (JDRP) at the federal level; and summarize some of the major issues 
that 310 projects should consider in their validation efforts. 

It's important that 310 projects consider the implications of the 
validation process from the initial stages of their implementation. Vali- 
dation of project success is not something that can be accomplished post 
hoc, after the project has run its course. Rather, careful, systematic 
planning, documentation, and evaluation are required throughout the life 
of the project. The Validation Guide, by providing an overview of factors 
that should be addressed, gives 310 projects a framework for guiding their 
validation efforts. 

While it is anticipated that the Validation Guide will be of most 
value to 310 projects, because the nature of their funding makes them 
particularly appropriate users, the contents of the Guide are such that it 
may be relevant and useful to any exemplary adult education project. 



RATIONALE FOR VALIDATION 

Reed (1981), in an historical review of the evolution of the 
validation movement, noted four early influences responsible for the 
phenomenon: 1) the need to justify substantial amounts of federal monies 
that had been pouring into local education agencies which would not con- 
tinue to be available in sizeable sums, 2) the need to justify the expen- 
ditures by demonstrating that the outcomes of federally funded LEA 
programs are generalizable, 3) the need to attest to the reliability of 
the programs and practices to be disseminated, and, 4) the limitations on 
the ability of national systems such as ERIC to effectively disseminate 
exemplary programs and practice s, A long range goal was to enable the 
creation of a bank of proven educational practices that provided some con- 
sumer protection for potential adapters or adopters. These factors led to 
the development of processes such as the Joint Dissemination Review Panel, 
the Identification, Validation and Dissemination process. Sharing Business 
Success (SBS), Project Information Packages (PIPs) , and the National 
Education Practices File. 

The rationale described above indicates a program sponsor's perspec- 
tive on the validation process, an important perspective certainly shared 
by NJDE and other state education agencies as the sponsors of adult educa- 
tion 310 projects. The National Association of State Advisory Councils 
(NASAC, 1979) described several other advantages of validation that com- 
plement the above list, while also addressing the perspectives of the 
developer and consumer. They note that the validation process provides a 
stimulus for structuring the development work on a beginning project in a 
systematic way, effectively marshalling the use of limited resources. The 




complexities of the development process are outlined and defined prior to 
adoption or adaptation. Forcing the publication of credible documentation 
about a project also reduces the risks to potential adopters and actually 
reduces overall developmental costs. The process insures that only 
"quality" projects will be disseminated. In addition, validation systems 
usually involve some type of network of projects that provides awareness, 
dissemination, and technical assistance. Finally, the NASAC study indi- 
cated that validation insures maximum educational benefit for students as 
well as cost efficiency for the development agency. 

Even in light of the rationales described above, a New Jersey 310 
Project might legitimately ask itself "How will the validation process 
benefit my 310 project?" A pamphlet produced by the U.S. Department of 
Education for the JDRP lists five important benefits that, although 
specifically written for the JDRP, are applicable to the validation 
process in general. These benefits are as follows. 

1- Internal Assessment - Evaluation data collected as part of the 
validation process give you effective tools for monitoring 
devel-^Dment and implementation as well as determining overall 
impact. 

2. External Assessment - Participation in a formal validation 

process gives you an objective, outside view of your project and 
can offer suggestions for improvement. 

3- Professional/Public Recognition - Successful validation brings 
recognition for your project's accomplishments from your col- 
leagues on local, state, regional, and national levels (depend- 
ing on the scope of the validation) and is an affirmation of 
your professional accomplishments. Approval also brings posi- 
tive recognition from the public for your school, or institu- 
tion, an important payoff in a time when education has been 
heavily criticized in the media. 

^- Increased Chances of Obtaining Funds - Successful validation 
improves your institution's track record. As educational 
budgets shrink, institutions with demonstrated effectiveness may 
be viewed as more efficient and less risky than funding untested 



agencies. In addition, the validated program is more likely to 
receive competitive funds. ^ 

^* g"try into Diffusion System - Through JDRP validation your 
project is eligible for entry into the National Diffusion 
Network (NDN) and may compete for dissemination funding. In 
such a case, your project could be used throughout the country. 

Your project receives the first two benefits listed above whether or 

not your validation effort is approved. The latter three are additional 

benefits that result from approved validation efforts. Closer to home. 

validated 310 projects could be disseminated across the state through 

an adult education 310 dissemination project. 
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CHARACTERISTICS OF VALIDATION MODELS 

As noted above, there are several different models of program 
validation that are currently being used. Reed, Patrick, and Holdzkom 
(1981) conducted a comprehensive survey to determine what validation 
processes were used by state education agencies. Virtually all of the 45 
states that reported a validation mechanism used either the IVD process, 
the JDRP process, a modification of one of these two, or a combination of 
IVD and JDRP. New Jersey falls into the latter category, using the basic 
IVD procedures but incorporating criteria from the JDRP. Although Nevj 
Jersey has suspended its formal validation process at present, Department 
officials are currently considering reopening validation procedures and 
instituting new validation initiatives. 

Several indispensable resource guides describe in detail the charac- 
teristics of validated programs, criteria for evaluation, steps for com- 
pleting the process, and examples of model submissions. These include the 
JDRP Ideabook (Tallmadge, 1977) and IVD's Sharing Educational Success 
(Hinze . 1979). In addition, specific evaluation concerns are addressed in 
two volumes published by the U.S. Office of Education in the mid- 
seventies, A Procedural Guide for Validating Achievement Gains in Educa- 
tion Projects (Tallmadge and Horst, 1976) and A Practical Guide to 
* Measuring Project Impact on Student Achievement (Horst, Tallmadge, and 

Wood, 1975). Finally, Reed's (1981) report for NIE's Re.iearch & Develop- 
ment Exchange, The Search for Quality Control in Dissemination of Educa- 
tional Products and Practices provides a comprehensive description of the 
various models for program validation. Although this Guide summarizes 

o 9 
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much of the information contained in the above documents, the original 
sources should be reviewed for more detailed description. 

There are proced'-\.dl differences between the validation models; there 
are also many similarities. Several common characteristics are listed 
^ below. 

• Detailed documentation of program background, development, and 
operations is needed. 

« 

• There is usually a focus on student impact. 

• Impacts mu^t be significant, that is, important and of suf- 
ficient magnitude. 

• Evidence of impact must be credible; "hard" evaluation data on 
sizable student samples is almost always required. 

• A sound experimental (or quasi-experimental) evaluation design 
is important in order to provide valid and reliable measures of 
post-intervention conditions and credible estimates of condi- 
tions without the intervention. 

• Programs must be able to be replicated or disseminated. 
Specific criteria and procedures are presented below for the JDRP and IVD 
validation processes. 

The Joint Dissemination Review Panel Process 

The JDRP, a joint effort of the former U.S. Office of Education and 
the National Institute of Education, was initiated in 1972 as a mechanism 
to review exemplary programs developed with federal funds so validated 
programs could be disseminated through systems such as the National Dif- 
fusion Network. The panel now reviews a broad range of programs from all 
states and supported through a varie*-y of developmental funding sources. 

The JDRP Ideabook indicates that "evidence of effectiveness is the 
sole criterion for approval.* It's important to emphasize that a program 
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must not only be effective, but must also provide credible evidence of 



effectiveness. This distinction is often problematic for many worthy 
program applicants. Six questions must be addressed in a JDRP applica- 
t-'on. 

1. Did a change occur? Was the change a positive one that was in 
some way related to the objectives? 

2. Was the effect consistent enough and observed often enough to be 
statistically significant? 

3. Was the effect educationally significant? In judging the edu- 
cational significance of an intervention's impact, two factors 
must be considered: the size of the effect and the importance 
of thr? area in which it happened. There ought to be a reason- 
able balance between the two factors. The chance that a small 
gain would be considered educationally significant is higher in 
a broad or educationally important area than in a narrow or less 
important area. 

4. Can the intervention be implemented in another location with a 
leasonable expectation of comparable impact? 

Is the project setting unique? 

Is the project effect solely due to the unique 
characteristics of the staff? 

What evidence is there to suggest that the intervention 
would work with different participants, in a different 
setting, and with a different staff? 

5. How likely is it that the observed effects resulted from the 
intervention? 

Can plausible alternative explanations be generated^ 
Can the alternative explanations be rejected? 

6. Is the presented evidence believable and interpretable? 

Are there any apparent inconsistencies in the data 
presented? 

Are enough data presented to satisfy the skeptical 
evaluator? 

Are the inferences drawn from the data consistent with the 
evidence? 



Has evidence been presented that common errors have been 
avoided? 

Project staff who decide to submit their project for review by JDRP 
follow a specific format for submitting materials (see Appendix). They 
can submit no more than 10 pages of explanation and documentation. Sur- 
veys of program applicants suggest that the difficulty of selecting the 
most appropriate information and of demonstrating avoidance of typical 
evaluation pitfalls lead most project directors to hire outside evalua- 
tors- 

Panel reviews examine factual accuracy, social fairness, and possible 
harm to users, as well as evidence of effectiveness. Approved projects 
are eligible for, but not guaranteed, dissemination funds distributed by 
the Department of Education for the National PL-fusion Network. Approved 
projects are described in the annual NDN catalog. Education Programs rh.r 
Work, which is distributed nationally. 

The Identification. V alidation. Dissemination (IVD) Process 

The IVD process was initiated by several national groups with heavy 
input from state education agencies. Although it was initially designed 
for the validation of Title III (and later Title IV-C) projects, the 
developers hoped that the procedures would be applicable for validating 
other projects as well. The IVD process is guided by a handbook titled 
Sharing Educational Succes s; A Handbook for Validation of Educational 

The handbook was written with the intention of allowing states 
a great deal of autonomy in the validation process. Using IVD, states 
validate exemplary projects based on the following two criteria. 

o Effectiveness/Success - Project objectives identified for 
validation are supported by convincing evidence showing 
statistically and educationally significant outcomes. The 
documented effectiveness or success of a program or practice is 
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of paramount importance for validation. A program or practice 
can be "proven to work" in numerous ways including: (1) by 
demonstrating with convincing evidence that the program will 
bring about desired change or improvement over existing prac- 
tices, (2) by demonstrating a more efficient or cost-effective 
program or practice through improved management, resource 
utilization, etc., or (3) by demonstrating with convincing evi- 
dence that a desired objective may be accomplished without 
detriment to the existing program. 

• Exportability Information is provided to demonstrate that the 
project or practice is capable of being diffused to other school 
districts and can be adopted or adapted by other school dis- 
tricts with similar needs and environments. For the project as 
a whole (or for each applicable component), information required 
includes evidence of educational significance, a description of 
the minimum level of adoption or replication which would produce 
similar results, and information about: the target population; 
staffing and training requirements; materials, equipment, and 
facilities; replication costs; and special problems. 

Sharing Educational Success describes six steps to be taken in the 

validation process. These steps can be modified by individual states to 

mee<" their own needs, and time-and-money-saving options are suggested. 

1. LEA completes and submits application for validation to the 
state agency or office responsible for coordinating validation 
activities at the state level. 

2. Preliminary review by the state agency for validation followed 
by: 

a. approval for validation team review; or 

b. return to the local education agency for revision according 
to suggestions; or 

c. disapproval for further validation. 

3. Selection of the validation team: 

a. the team leader to be selected out-of-state from the list 
of U.S. Department of Education's trained team leaders 
within the region, 

b. two team members selected within the state from the list of 
state-trained members. 

4. Review of application by individual team members. (This may be 
done as individuals in isolation from the other team members or 
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the team may be convened to review an individual 
application or serve as a panel to review a group of 
applications. ) 



a. the team leader makes recommendations to the state agency 
responsible for validation that the identified revisions 
are to be completed before the on-site team visit is 
conducted. 

b. the team leader informs the state agency responsible for 
validation that the on-site visit is to be conducted 
according to the existing application, 

5. Conduct of the on-site visit. The most frequent procedure is to 
send all team members on-site. Some states have designated an 
individual team member to conduct the on-site visit as a cost 
saving measure. The individual conducting the on-site visit 
serves the role of collecting and clarifying any incomplete or 
missing information identified by the review of the application 
by the whole team. 

6. The primary decision of the validation team is either approval 
or disapproval under the IVD standards. 

If approved, the team might also make the following 
recommendations : 

a. submission to JDRP. 

b. state dissemination. 

c. a special component or product be recognized as worthy of 
distribution. 

The IVD process requires no commitments from the federal or state 
governments. In the past, many states have operated their own diffusion 
networks and have offered dissemination funding. In addition, the IVD 
process is often a preliminary step in preparing a JDRP submission. 
Specific submission requirements for the IVD process are summarized in the 
Appendix. 

Evaluating for Validation 

To prepare properly for validation of any project, a car^i-fully 
planned and implemented evaluation is required. For a detailed discussion 
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of evaluation issues in adult education, see the companion document to 
this one entitled An Evaluation Guide for Adult Education Projects (Dusewicz, 
Biester and Kenney, 1983). While the companion document referenced above 
addresses evaluation planning, design and procedures in depth, this sec- 
tion of the present Guide presents a brief overview of evaluation pitfalls 
and suggestions related to validation. 

Many JDRP applications are rejected because of flaws in the evalua- 
tion design. The KMC Research Corporation, in a major effort to validate 
Project Information Packages (PIPs) for exemplary compensatory education 
programs, identified 14 common flaws or potential pitfalls in conducting 
an evaluation study (Horst, Tallmadge, and Wood, 1975). The JDRP Ideabook 
incorporates this list of evaluation hazards and describes how to avoid 
them. The flaws are listed below. 

1. Claiming much, providing evidence of little. Where evidence 
matches the claims of what a project says it can do, a favorable 
decision is far more likely than where evidence falls far short 
of goals, objectives, and claims. 

2. Selecting measures not logically related to the intervention. 

3. The use of grade-equivalent scores. Grade-equivalent scores 
provide an insensitive, and, in some instances, a systematically 
distorted assessment of cognitive growth. As such, the JDRP 
does not regard them as credible indicators of achievement or 
growth. 

4. The use of a single set of test scores for both selecting and 
pretesting participants. 

5. The use of comparisons with inappropriate test dates for 
obtaining information. In norm-referenced evaluations, tests 
should be administered at nearly the same time as the test 
publisher tested the norm group. 

6. The use of inappropriate levels of tests. 

7. Missing data. 

8. The use of noncomparable treatment and control groups. 
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9. The use of inappropriate statistical adjui=Jtinents with non- 
equivalent control groups. Making between-group comparisons 
using either "raw" gain scores or "residual" gain scores should 
be assiduously avoided. 

10. Constructing a matched control group after the treatment group 
has been selected. 

11. The careless collection of data. 

12. The use of different instruments for pretesting and posttesting. 

13. The use of inappropriate formulas to generate no-treatment 
expectations. Many projects use an unrealistic theoretical 
model or foimula to calculate "expected" posttest scores from IQ 
or other pretest scores. If students do better than the 
calculated expectation, the project is considered a success. 

14. Mistaken attribution of causality. The plausibility of 
alternative explanations should be carefully examined before 
evaluation results are attributed to project impact, as 
evaluation hazards are often the cause of apparent gains or 
losses. 

It is important to consider these potential pitfalls in the project 
planning stages. If project administrators and staff do not understand 
the issues in dealing with such pitfalls, the JDRP recommends hiring an 
evaluation consultant. However, hiring an evaluation specialist after the 
fact cannot eliminate such flaws. 

RBS' experience with the JDRP and IVD processes has suggested a few 
other recommendations for preparing a validation application, as follows. 

1. Panel members will not spend a lot of time in laboring to 
understand the essential elements of your submission. Essential 
points need to be stated in a clear, concise way. 

2. Regarding the above point, consider the use of subheadings that 
directly reflect the panel's criteria for assessing evidence of 
effectiveness (JDRP criteria are interpretability, credibility, 
evidence of impact, statistical reliability, educational sig- 
nificance, internal validity, and external validity). Speci- 
fically noting these under explicit subheadings would help 
assure the reader that you have attended to all concerns. 

3. In addition to being a technical document, the submission is 
also a marketing document. You need to "sell" the reader on the 
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idea that your project is important, effective, and worth being 
disseminated as exemplary. 



4. Make a case for your project's uniqueness right up front. 
Unique solutions to common problems stand a better chance for 
acceptance. 

5. Clearly indicate your claims of effectiveness. The "evidence" 
section shows why you are making each claim. Evidence must be 
credible . 

6. Sample size and comparison groups are often problems with vali- 
dation of adult basic education programs. A strategy sjjccess- 
fully employed by the F.I.S.T. project in New Brunswick was to 
use a replication design where the evaluation study was con- 
ducted twiQe. Similarly positive evaluation results for 
different samples at two points in time proved to be convincing 
evidence for JDRP despite small sample sizes and the lack of a 
comparison group. 

7. Don't make claims for outcomes where there is no evidence, 

8. Consider the documentation of side effects as you plan your 
evaluation study. 

9. Remember that panel members, for the most part, tend to have an 
"experimental psychology" perspective when it comes to 
evaluation. As such, you'll need to use the jargon of 
experimental design. 

10. Talk to someone familiar with the JDRP process before going to 
Washington for your final panel review. The experience can 
often be an intimidating and frustrating one for those who don't 
know what to expect. 

11. Don't be discouraged if the initial application is not accepted. 
Valuable feedback and experience will have been gained and 
reapplications are often successful. 



A JDRP-validated project operated through Middlesex College; evaluation 
assistance was received from Rutgers University, and from RBS. 
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OTHER ISSUES IN VALIDATION 

Reed (1981) and others have noted a number of other issues related to 
the validation process. Six of these are briefly discussed below. 

Adoption vs. Adaption 

Research on Innovation and change has shown that exemplary programs, 
\Aien replicated, most often are adapted by local education agencies rather 
than adopted. That Is, they are changed or modified to suit local condi- 
tions and needs, rather than replicated exactly as the developer intended. 
Although the emphasis of validation panels Is clearly on "outcome evalua- 
tion," the above finding Illustrates the need for a process evaluation of 
the project. Operations need to be clearly documented. In detail, using 
qualitative and quantitative evaluation techniques. 

Evaluation Experience of Project Staff 

Most development project staff lack the technical evaluation exper- 
tise needed to address design and analysis concerns in the validation 
process. Outside help is frequently recommended. However, this recom- 
mendation has budgetary implications that need to be addressed in the 
project planning stage. 

Exemplary vs. Promising Programs 

The validation process, with its emphasis on hard evaluation data, is 
a rigorous one. Often, programs that are very effective in achieving 
highly desirable outcomes cannot go through the validation process suc- 
cessfully because of constraints such as feasibility or budgetary 
limitations. Thus, uany promising practices are not publicized through 
existing dissemination channels. Recently, mechanisms have been developed 
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to address these omissions, such as the National Educational Practices 
File being compiled through the ERIC system, and other computerized 
knowledge bases such as SPIF/SPIN developed by Bibliographic Retrieval 
Services (BRS) . 

Educational vs. Statistical Significance 

Validation panels usually require two types of significance — 
statistical and educational. Although they are somewhat related, evalua- 
tion results could demonstrate either type by Itself. That Is, a statis- 
tically significant outcome may not be educationally significant and vice 
versa. Statistical significance, as the name implies, involves the use of 
inferential statistics to determine that outcomes can be attributed to 
something other than chance probabilities. Educational significance, on 
the other hand. Is a judgment of whether an Intervention's Impact is 
sizable and Important. In JDRP's words, "An educationally significant 
effect Is one of nontrlvlal magnitude. In a content area generally 
accepted as Important, which can be achieved at a reasonable cost." Edu- 
cational significance Is determined through theory, past experience, and 
expert judgment. In addition, statistical rules of thumb are often 
applied (e.g., the size of a change must exceed one-third of a standard 
deviation) . 

Responsibilities of Validated Program Staff 

Staff members should realize that their roles may change as a result 
of successful validation. Responsibilities typically Include answering 
requests, preparing brochures or other documents describing the project, 
scheduling and handling visitors, and participating In other dissemination 
activities such as educational fairs or conferences. In addition, staff 
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could become heavily Involved in providing training and technical assis- 
tance in helping other agencies to replicate/adapt their program. Tn some 
cases, such as NDN, funds are available to support these services. How- 
ever, applicants should realize that success has its price and the above 
responsibilities need to be considered when beginning the validation 
process. 

Costs of Validation 

Although there are clear benefits in conducting a thorough evalua- 
tion, and the rationale for participating in a validation effort is a 
convincing one, certain costs will be incurred. This is particularly true 
when an evaluation consultant is employed. Comprehensive evaluation 
efforts can be quite costly, A typical rule of thumb in preparing cost 
estimates for development efforts is to allocate ten percent of the 
project budget for evaluation tasks. However, in a time of diminishing 
resources, important decisions on the best use of limited funds must be 
made. The NASAC survey of IVD submissions indicated that Title IV-C 
projects in 1979 typically spent between $1,500 and $6,000 on evaluation 
activities. Project managers need to consider these costs during the 
initial planning stages. 
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CONCLUSIONS 

This Guide has provided a brief overview of the issues involved in 
the validation process. Although the benefits of going through the 
process are many, so are the potential pitfalls. Careful planning and 
attention to the basic principles of good educational evaluation can lead 
to a successful validation experience. Validation of high quality educa- 
tional programs helps to insure that practices which are disseminated to a 
wide audience in a cost-effective way really fulfill the claims which are 
made for them. The ultimate beneficiaries of the validation process are 
the students who participate in these exemplary efforts during the devel- 
opment period and in succeeding generations as the project multiplies 
through successful replications at additional sites. 
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BEST COPY AVAILABLE 



FORMAT FOR SUBHiniNG MATERIALS 
TO THE JOINT OISSEMIHATION REVIEW PANEL 



PROGRAM AREA: ^JJ^^.j Jlt^MII . mding, ciretr tducitlon, environmental education, education for the 



I. INTERVENTION TITLE, LOCATION: 

Sltted!" °' Intervention and the location for which evidences of effectiveness Is being 



II. DEVELOPED BY: 

Indicate who developed the Intervention originally, even If this happened at a different utm th^n f-K- 
one for which evidence of effectiveness Is being presented. "^pp''"^ • airrerent site than the 

III.- SOURCE AND LEVEL OF FUNOING: 

List all funding sources for the Intervention at the location for which evidence of «ffectlv.n.« u 
presented and, for each source, list the amount of funds (sea Figure 1 for wexampU^^ ' 

IV. YEARS OF INTERVENTION DEVELOPMENT: 

Indicate the year or years during which the intervention was originally developed or tested. 

V. BRIEF DESCRIPTION OF INTERVENTION: 

What Is the Intervention? 

What anj Its objectives? 

What clalfflj of effectiveness are being made? 

What If the context In which It operates? 

Who are the Intended users and beneficiaries? 

What are the costs of adoption and maintenance of the Intervention? 
VI. EVIDENCE OF EFFECTIVENESS: 

ZV.r^oX'-X'ii: '^^'^^^r^ <" 



Evidence that the quantitative measures ara reliable and valid 



InterpretablHty of meaxtirgs 
Jnakators of the effects claimed. 

VuLk*"?' TJll^"''^]* " th» 1nt,rv.nt.<on. C.n..lt.m.tlv9 axpl.natlon, 

practice eff.cts. iMt^r.tion, selection of* sup«Hor tr..tracnt group., «g.. bt nilwl out? 
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IVO SUBMISSION REQUIREMENTS 
(Abbreviittd Form) 

Part 1. Informatinn and Ovgrv^iiw 
A. Applicant tnfonnatlori {including txpenditurts) 
Project Abstract or .Overview 

»:jTs.;f;rM ±:s;„j;r?s; c;;;.:- « - ...... 

Part ir. gffectivi>ness/Suee<»<< 
A. Purpose and Ob.lectivet 

B- P^^qrani Activ^»^>« 



e. Rel lability 

d. Nonjf group (if nora-rsfereneed tests) 

°- Insults and *n«1^>,|^ 

H. renewing ,on.t .u,d b. H«,p,u, in .spending ,or ..cH obj.ctiv.: " 

For overin p„ject resulti: 

aHirJn'tSr'pSt";;?^^";; - S,-""' <" "ndmg, 

3.' " """""""^ — 

".port u„.nticip.t,. outco^s 0, «Jo. 
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BEST COPY AVAILABLE 



BEST COPY AVAILABLE 



IVD SUBMISSION REQUIREMENTS 
(Abbreviated Fonr.) 
Continued 



Pert in. Exportablllty 

The documented sr cess of a program Is an essential part In making that program available for diffusion 
IJM^Jw ^^S?"^"* of diffusion may require a different, but related, set of program activities and 
waterlals. This section will Identify and document the program's capability to diffuse a successful program. 

1. Educational Significance 



Importance to the educational community, magnitude of the problem, benefits of a replication In 
another school site, etc. 

Z, Target Population 



Describe _ 
characterl 



the appropriate learner population for the rcollcatlon of the program and any unique 
Istlcs about the original site that may limit the success of a replication. 



3. Staffing and Training Requirements 

Describe special staffing and any training that Is needed In order to replicate the program. Is such 
sUff usually available to a school district; can the training be segmented? « * • 

4. M aterials, Equipment, Facilities 

Describe all required program materials, equipment, and facilities necessary to replicate the 
program. Provide copies during on*slte visit. 

5- M inimum Adoption or Replication 

Describe what would constitute a Bilnlmum level of replication of your program that would produce 
similar results to those you have documents as successful. Can Individual components be 
rcpl lea ted? 

6. Replication Costs 

Detail all costs. Including costs of training, materials, and start-up. 

7. Special Problems 

Describe special problems that are likely to be encountered In the replication and operation of your 
program. How can they be overcome or avoided? ^ 
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